Multi-layered Spiking Neural Network with Target Timestamp Threshold Adaptation and STDP

Spiking neural networks (SNNs) are good candidates to produce ultra-energy-efficient hardware. However, the performance of these models is currently behind traditional methods. Introducing multi-layered SNNs is a promising way to reduce this gap. We propose in this paper a new threshold adaptation system which uses a timestamp objective at which neurons should fire. We show that our method leads to state-of-the-art classification rates on the MNIST dataset (98.60%) and the Faces/Motorbikes dataset (99.46%) with an unsupervised SNN followed by a linear SVM. We also investigate the sparsity level of the network by testing different inhibition policies and STDP rules

Unsupervised Visual Feature Learning with Spike-timing-dependent Plasticity: How Far are we from Traditional Feature Learning Approaches?

Spiking neural networks (SNNs) equipped with latency coding and spike-timing dependent plasticity rules offer an alternative to solve the data and energy bottlenecks of standard computer vision approaches: they can learn visual features without supervision and can be implemented by ultra-low power hardware architectures. However, their performance in image classification has never been evaluated on recent image datasets. In this paper, we compare SNNs to auto-encoders on three visual recognition datasets, and extend the use of SNNs to color images. The analysis of the results helps us identify some bottlenecks of SNNs: the limits of on-center/off-center coding, especially for color images, and the ineffectiveness of current inhibition mechanisms. These issues should be addressed to build effective SNNs for image recognition

S3TC: Spiking Separated Spatial and Temporal Convolutions with Unsupervised STDP-based Learning for Action Recognition

Video analysis is a major computer vision task that has received a lot of attention in recent years. The current state-of-the-art performance for video analysis is achieved with Deep Neural Networks (DNNs) that have high computational costs and need large amounts of labeled data for training. Spiking Neural Networks (SNNs) have significantly lower computational costs (thousands of times) than regular non-spiking networks when implemented on neuromorphic hardware. They have been used for video analysis with methods like 3D Convolutional Spiking Neural Networks (3D CSNNs). However, these networks have a significantly larger number of parameters compared with spiking 2D CSNN. This, not only increases the computational costs, but also makes these networks more difficult to implement with neuromorphic hardware. In this work, we use CSNNs trained in an unsupervised manner with the Spike Timing-Dependent Plasticity (STDP) rule, and we introduce, for the first time, Spiking Separated Spatial and Temporal Convolutions (S3TCs) for the sake of reducing the number of parameters required for video analysis. This unsupervised learning has the advantage of not needing large amounts of labeled data for training. Factorizing a single spatio-temporal spiking convolution into a spatial and a temporal spiking convolution decreases the number of parameters of the network. We test our network with the KTH, Weizmann, and IXMAS datasets, and we show that S3TCs successfully extract spatio-temporal information from videos, while increasing the output spiking activity, and outperforming spiking 3D convolutions

Planifier les affectations spatio-temporelles d’autrui : l’articulation d’enjeux économiques et sociaux par des ordonnanceurs

La construction des plannings de travail, aussi appelée ordonnancement du personnel, soulève de multiples enjeux, tant économiques que commerciaux ou encore sociaux. Elle représente notamment un levier pour construire des conditions de travail satisfaisantes. Au lieu de s’intéresser aux impacts de l’absentéisme sur l’activité des agents présents au travail, comme fréquemment discuté dans la littérature ergonomique, le but de cet article est de présenter l’activité de conception de ces plannings de travail, c’est-à-dire celle d’ordonnanceurs d’une grande entreprise ferroviaire dans un contexte de flexibilité et de compétitivité accrues.À partir de l’analyse de leur activité, nous montrons comment les ordonnanceurs développent un travail d’articulation à la fois entre les différents acteurs concernés par les plannings, mais également entre les différentes dimensions de la production (économique, commerciale, sociale…) afin de construire des plannings acceptables et acceptés. Ce travail d’articulation constitue alors un levier pour la gestion de l’absentéisme et la prévention des troubles de santé chez ces opérateurs.Conjointement ces régulations sont mises au regard avec l’indicateur de pilotage utilisé par l’organisation pour rendre compte de cette activité d’ordonnancement. Le contraste des analyses d’activité avec celles issues des indicateurs de suivi mis en place au sein de l’unité montre que ces articulations sont souvent masquées, et peuvent se montrer en contradiction avec les décisions de la direction.The construction of work schedules, also known as personnel scheduling, raises several economic, commercial and social issues. In particular, it represents a lever for building satisfactory working conditions. Instead of focusing on the impacts of absenteeism on the activity of agents present at work, as frequently discussed in the ergonomic literature, the purpose of this article is to present the activity of designing these work schedules, i.e. that of schedulers in a large railway company in a context of increased flexibility and competitiveness.Based on the analysis of their activity, we show how schedulers develop a work of articulation both between the different actors concerned by the schedules, and between the different dimensions of production (economic, commercial, social, etc.) in order to build acceptable and accepted schedules. This articulation work thus constitutes a lever for managing absenteeism and for preventing health problems among the operators in question. At the same time, these regulations are compared with the management indicator used by the organization to report on this scheduling activity. The contrast between the activity analyses and those resulting from the monitoring indicators set up within the unit shows that these links are often hidden and can be inconsistent with management decisions

Traitement automatique des langues pour l'indexation d'images

Although it is globally in line with traditional information retrieval (IR), image indexing makes poor use of the existing work about textual IR and natural language processing (NLP). We identify two levels where such work could become integrated to image indexing systems. The first level is the description of the visual content of images. To integrate NLP at this level, we adopt a visual word-based representation of images, as proposed by Sivic and Zisserman. This representation raises two issues that are classical in textual IR: choosing relevant index terms and taking into account the relations between index terms. We address the first issue by studying stop-lists and weighting schemes in the context of image indexing. Our experiments show that there is no optimal weighting scheme in the general case, and that it should be chosen in keeping with the query. Then, we address the second issue by adapting language models to images, to go beyond the term independence hypothesis. Our experiments show that, in the context of image classification, taking account of spatial relations between visual words can improve the systems' performances. The second level where we integrate NLP to image indexing is semantic image indexing: we can use NLP techniques on texts coming with images to extract a textual description of these images. We first show that standard image descriptors are not suited to image annotation, then we propose an image annotation scheme that avoid this problem by using high-level textual and visual concepts: we extract named entities from texts and associate them with visual concepts that we detect in the images. We validate our approach on a real-world and large-scale news corpus.Bien que s'inscrivant dans un cadre global de recherche d'information (RI) classique, l'indexation d'image ne tire que peu parti des nombreux travaux existants en RI textuelle et en traitement automatique des langues (TAL). Nous identifions deux niveaux auxquels de tels travaux peuvent s'intégrer aux systèmes d'indexation d'images. Le premier niveau est celui de la description du contenu visuel des images. Pour y intégrer des techniques de TAL, nous adoptons la description des images par mots visuels proposée par Sivic et Zisserman. Cette représentation soulève deux problématiques similaires aux problématiques classiques de la RI textuelle~: le choix des termes d'indexation les plus pertinents pour décrire les documents et la prise en compte des relations entre ces termes. Pour répondre à la première de ces problématiques nous proposons une étude des stop-lists et des pondérations dans le cadre de l'indexation d'images. Cette étude montre que, contrairement au cas des textes, il n'existe pas de pondération optimale pour tous types de requêtes, et que la pondération doit être choisie en fonction de la requête. Pour la seconde, nous utilisons des modèles de langues, outil classique du TAL que nous adaptons au cas des images, pour dépasser l'hypothèse d'indépendance des termes dans un cadre de classification d'images. Nos expérimentations montrent que prendre en compte des relations géométriques entre mots visuels permet d'améliorer les performances des systèmes. Le second niveau étudié est l'indexation sémantique des images : il est possible d'utiliser des méthodes de TAL sur des textes accompagnant les images pour obtenir des descriptions textuelles de celles-ci. Dans un premier temps, nous montrons que les descripteurs classiques d'images ne permettent pas d'obtenir des systèmes d'annotation d'images efficaces. Puis nous proposons une méthode d'annotation qui contourne cet écueil en se basant sur des descripteurs textuels et visuels de haut-niveau~: nous extrayons des textes des entités nommées, que nous mettons en relation avec des concepts visuels détectés dans les images afin d'annoter celles-ci. Nous validons notre approche sur un corpus réel et de grande taille composé d'articles de presse

Introducing FoxPersonTracks: A benchmark for person re-identification from TV broadcast shows

International audienceThis paper introduces a novel person track dataset dedicated to person re-identification. The dataset is built from a set of real life TV shows broadcasted from BFMTV and LCP TV french channels, provided during REPERE challenge. It contains a total 4,604 persontracks (short video sequences featuring an individual with no background) from 266 persons. The dataset has been built from the REPERE dataset by following several automated processing and manual selection/filtering steps. It is meant to serve as a benchmark in person re-identification from images/videos. The dataset also provides re-identifications results using space-time histograms as a baseline, together with an evaluation tool in order to ease the comparison to other re-identification methods

Improving STDP-based Visual Feature Learning with Whitening

International audienc

Improving STDP-based Visual Feature Learning with Whitening

International audienc

Traitement automatique des langues pour l'indexation d'images

Nous nous intéressons, dans cette thèse, à l'usage du traitement automatique des langues (TAL) dans les systèmes d'indexation d'images. Au niveau de la description du contenu visuel des images, nous nous appuyons sur la description des images sous forme de mots visuels, qui pose des problématiques similaires à celles de l'indexation textuelle. Nous utilisons des méthodes de TAL (pondérations et stop-lists) afin de déterminer les mots visuels pertinents, puis nous utilisons les modèles de langues pour prendre en compte certaines relations géométriques entre mots visuels. Au niveau de la description du contenu sémantique des images, nous proposons une méthode d'annotation d'images basée sur l'extraction d'entités nommées pertinentes dans des textes accompagnant les images à annoter.In this thesis, we propose to integrate natural language processing (NLP) techniques in image indexing systems. We first address the issue of describing the visual content of images. We rely on the visual word-based image description, which raises problems that are well known in the text indexing field. First, we study various NLP methods (weighting schemes and stop-lists) to automatically determine which visual words are relevant to describe the images. Then we use language models to take account of some geometrical relations between the visual words. We also address the issue of describing the semantic content of images: we propose an image annotation scheme that relies on extracting relevant named entities from texts coming with the images to annotate.RENNES1-BU Sciences Philo (352382102) / SudocRENNES-INRIA Rennes Irisa (352382340) / SudocSudocFranceF